Method and system for data fusion using spatial and temporal diversity between sensors

ABSTRACT

A method and system provide a multi-sensor data fusion system capable of adaptively weighting the contributions from each one of a plurality of sensors using a plurality of data fusion methods. During a predetermined tracking period, the system receives data from each individual sensor and each data fusion method is performed to determine a plurality of reliability functions for the system based on combining each sensor reliability function which are individually weighted based on the S/N (signal-to-noise) ratio for the received data from each sensor, and a comparison of predetermined sensor operation characteristics for each sensor and a best performing (most reliable) sensor. The system may dynamically select to use one or a predetermined combination of the generated reliability functions as the current (best) reliability function which provides a confidence level for the multi-sensor system relating to the correct classification (recognition) of targets and decoys.

CROSS-REFERENCE

[0001] This application claims the benefit of U.S. provisionalapplication Ser. No. 60/367,282, filed Mar. 26, 2002.

TECHNICAL FIELD

[0002] The present invention relates generally to data fusion. Itparticularly relates to a data fusion method that dynamically selects atleast one fusion technique to integrate data from a plurality of sensorshaving spatial and temporal diversity.

BACKGROUND OF THE INVENTION

[0003] Sensor systems incorporating a plurality of sensors (multi-sensorsystems) are widely used for a variety of military applicationsincluding ocean surveillance, air-to-air and surface-to-air defense(e.g., self-guided munitions), battlefield intelligence, surveillanceand target detection (classification), and strategic warning anddefense. Also, multi-sensor systems are used for a plurality of civilianapplications including condition-based maintenance, robotics, automotivesafety, remote sensing, weather forecasting, medical diagnoses, andenvironmental monitoring (e.g., weather forecasting).

[0004] To obtain the full advantage of a multi-sensor system, anefficient data fusion method (or architecture) may be selected tooptimally combine the received data from the multiple sensors. Formilitary applications (especially target recognition), a sensor-levelfusion process is widely used wherein data received by each individualsensor is fully processed at each sensor before being output to a systemdata fusion processor. The data (signal) processing performed at eachsensor may include a plurality of processing techniques to obtaindesired system outputs (target reporting data) such as featureextraction, and target classification, identification, and tracking. Theprocessing techniques may include time-domain, frequency-domain,multi-image pixel image processing techniques, and/or other techniquesto obtain the desired target reporting data.

[0005] An exemplary, prior art example of a multi-sensor, sensor-levelfusion (process) system 100 for automatic target recognition (ATR) isshown in FIG. 1. Advantageously, system 100 may include a plurality ofsensors 102, 104, 106, 108 which may include RF sensors such as MMWradar (active sensor) 102, MMW radiometer (passive sensor) 104, IR laserradar 106, and passive IR sensor 108 (e.g., FLIR or IRST—infrared searchand track). Additionally, multi-sensor system 100 may include dataprocessor portion 118 which includes sensor parallel processor 120 anddata fusion processor 122 which advantageously executes at least onepredetermined algorithm to produce a valid target declaration output124. Each sensor may scan a predetermined area (field of view) for anobject (target) and receive data using antenna 110 (for the MMW sensors102,104) or lens 114,116 (for IR sensors 106, 108). In accordance withthe sensor-level fusion architecture selected, each sensor may have itsindividually received data processed (via parallel processor 120) usingthe predetermined algorithm that may be designed in accordance with aplurality of predetermined system parameters including receivedfrequency band, active or passive operating mode of the individualsensor, sensor resolution and scanning characteristics, target andbackground signatures, and other predetermined system parameters.Results of the individual sensor processing may be input as a targetreport to the data fusion processor 122 (in response to a cue/query fromthe data fusion processor) where the results may be combined (fused) inaccordance with the predetermined algorithm to produce an outputdecision 124 such as “validated target” or “no desired targetencountered”. Other output decisions 124, such as tracking estimates,may be produced in accordance with multi-sensor system outputrequirements. The tracking estimates may be used to form new trackingresults, update existing tracking, and estimate future positions of theobject (target).

[0006] Many multi-sensor systems (such as system 100 in FIG. 1) usefeature-level fusion wherein features that help discriminate (find smalldistinctions) among objects (targets) are extracted from each individualsensor's data and then combined to form a composite feature vectorrepresentative of the object in each sensor's field of view. Thecomposite feature vector may be input to a data processor (or neuralnetwork) and classification (recognition of the object as a house, tank,truck, man, etc.) of the object may then occur using a predeterminedalgorithm (incorporating the previously described processing techniques)to recognize the object of interest, differentiate the object fromdecoys (false targets), and produced a weighted value (e.g., reliabilityvalue) that links the observed object to a particular (predetermined)target with some probability, confidence, threat priority, or othercategorical parameter.

[0007] Currently, feature-level, multi-sensor systems exclusively useone of a wide variety of data fusion methods (strategies) which mayinclude multiplicative fusion (e.g., Bayes or Dempster-Shafer methods),data fusion using fuzzy logic (e.g., min, max calculations), or anotherdata fusion method. The use of only a single data fusion method mayreduce the confidence (reliability or probability) level of the systemoutput since a different data fusion method (or the combination ofdifferent methods with the current method) may generate a higher (moreoptimum) reliability level for the plurality of sensors (which may havedifferent sensor reliability levels over different tracking periods dueto different sensor constraints, atmospheric conditions, or otherfactors) and thus may produce a less accurate data fusion output (targetclassification) when using only a single data fusion method.Additionally, under certain conditions, a data fusion reliability output(using data from all sensors) may be worse than a single sensorreliability output.

[0008] Therefore, due to the disadvantages of the current multi-sensorsystem using only a single data fusion method, there is a need toprovide a multi-sensor system that adaptively weights the contributionsfrom each sensor using a plurality of data fusion methods. The systemmay perform each data fusion method to generate a plurality ofreliability functions for the plurality of sensors, and then dynamicallyselect to use one, or a predetermined combination, of the generatedreliability functions as the current (best) reliability function forimproved reliability of system target classification. Also, there is aneed to provide a multi-sensor data fusion system that can dynamically(adaptively) switch to a single sensor reliability output whenpredetermined conditions arise making the single sensor output betterthan a data fusion output.

SUMMARY OF THE INVENTION

[0009] The method and system of the present invention overcome thepreviously mentioned problems by providing a multi-sensor data fusionsystem capable of adaptively weighting the contributions from each oneof a plurality of sensors using a plurality of data fusion methods.During a predetermined tracking period, the system receives data fromeach individual sensor and each data fusion method is performed todetermine a plurality of reliability functions for the system based oncombining each sensor reliability function which are individuallyweighted based on the S/N (signal-to-noise) ratio for the received datafrom each sensor, and a comparison of predetermined sensor operationcharacteristics for each sensor and a best performing (most reliable)sensor. The system may dynamically select to use one or a predeterminedcombination of the generated reliability functions as the current (best)reliability function to provide a confidence level for the multi-sensorsystem relating to the correct classification (recognition) of targetsand decoys.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010]FIG. 1 is a block diagram of an exemplary sensor-level data fusionsystem found in the prior art;

[0011]FIG. 2 is a functional block diagram of an exemplary sensor-leveldata fusion system in accordance with embodiments of the presentinvention.

[0012]FIGS. 3a-3 d show diagrams of exemplary sensor classificationresults for a multi-sensor system using a plurality of differenttemporal fusion methods in accordance with embodiments of the presentinvention.

[0013]FIGS. 4a-4 d show diagrams of exemplary sensor classificationresults for a multi-sensor system using different spatial fusion methodsin accordance with embodiments of the present invention.

[0014]FIGS. 5a-5 h show diagrams of exemplary sensor classificationresults for a multi-sensor system using additive and multiplicativefusion in accordance with embodiments of the present invention.

[0015]FIGS. 6a-6 d show diagrams of exemplary sensor classificationresults for a multi-sensor system using multiple fusion methods inaccordance with embodiments of the present invention.

[0016]FIG. 7 shows a flowchart of an exemplary data fusion process inaccordance with embodiments of the present invention.

DETAILED DESCRIPTION

[0017]FIG. 2 shows a functional block diagram of an exemplarymulti-sensor, sensor-level data fusion system 200 in accordance withembodiments of the present invention. Advantageously, multi-sensorsystem 200 may include sensor component 206, adaptive processor 207, anddata fusion (integration) selection processor 208. Sensor component 206may include a plurality of sensors 205 (and associated sensorprocessors) to receive and compute data from an object (target) within apredetermined scanning area (field of view) where the scanning data mayinclude acoustic, electromagnetic (e.g., signal strength,SNR—signal-to-noise ratio, etc.), motion (e.g., range, direction,velocity, etc.), temperature, and other types ofmeasurements/calculations of the object scanning area. It is noted thatthe FIG. 2 illustration of adaptive processor 207 and data fusionselection processor 208 as separate components is solely exemplary, andshould not be viewed as a limitation upon the present invention as thetwo components may be combined into a single component and still bewithin the scope of the present invention.

[0018] The plurality of sensors 205, using associated sensor processors,may each perform the well-known process of feature extraction to detectand pull out features which help discriminate the objects in eachsensor's field of view and combine all the feature extractions (fromeach sensor) as a composite input to adaptive processor 207. Operatingin combination, adaptive processor 207 and data fusion selectionprocessor 208 may perform, as described in detail later, all levels ofdiscrimination (detection, classification—recognition, identification,and tracking) of the object (target) using at least one predeterminedalgorithm (e.g., data fusion) to recognize the object of interest,differentiate the object from decoys (false targets), and produce atleast one (or a predetermined combination of two or more) weighted,(system) reliability function that links the observed object to apredetermined target with some confidence level. The system reliabilityfunction may be used to generate a decision output 210 (target report)for target detection such as “validated target” or “no desired targetencountered”. Also, alternatively, plurality of sensors 205 mayfeed-through (without processing or with minimal processing) receiveddata to processors 207, 208 for feature extraction and targetdiscrimination processing.

[0019] The particular combination of sensors 205 for system 200 mayinclude a number of different sensors selected to provide exemplarypredetermined system attributes (parameters) including temporal andspatial diversity (fusion), sensitivity, bandwidth, noise, operatingrange, transmit power, spatial resolution, polarization, and othersystem attributes. These different sensors may include, but are notlimited to, passive and/or active sensors operating in the RF (radiofrequency) range such as MMW (millimeter-wave) sensors, IR (infrared)sensors (e.g., Indium/Antimony—InSb focal plane array), laser sensors,and other passive and/or active sensors useful in providing theexemplary predetermined system attributes.

[0020] During exemplary operation as described herein and in accordancewith the flow process diagram shown in FIG. 7, at step 702, each one ofthe plurality of (differently located) sensors 205 may receive andcalculate (compute) data about the object during a predetermined time(tracking) period over a plurality of time frames to provide spatial andtemporal diversity for system 200. The computed data may include signalmeasurements (e.g., noise, radiance, reflection level, etc.) that areused to determine SNR (signal-to-noise ratio) for each sensor during thepredetermined tracking period. Thereafter, at step 704, the computed SNRfor each one of the plurality of sensors 205 may be used by adaptiveprocessor 207 to generate a plurality of sensor reliability functionsfor each sensor. Each different sensor reliability function (generatedfor each sensor) may be produced by using each one of a plurality ofdifferent fusion methods. The plurality of different fusion methods mayinclude, but are not limited to, additive fusion, multiplicative fusion(e.g., Bayes and/or Dempster-Shafer), fuzzy logic fusion using minimumand/or maximum calculations, and other fusion methods (strategies) thathelp to minimize the errors associated with noise. Likelihood function(pdf) calculations for each fusion method are shown in Appendix A.

[0021] Following step 704 of generating individual sensor reliabilityfunctions using each different fusion method, at step 706, adaptiveprocessor 207 (in accordance with a predetermined algorithm usingadaptive weighting as described in detail later) may generate aplurality of overall (combined) reliability functions for system 200 foreach different fusion method. The plurality of generated systemreliability functions may be input to data fusion selection processor208. Then, at step 708, fusion selection processor 208 may select one ora predetermined combination of the plurality of combined reliabilityfunctions as the current (best) reliability function for system 200.Thereafter, at step 710, data fusion may be immediately performed (ordelayed to a subsequent, predetermined time frame) and a decision output(target report) may be generated using the combined system reliabilityfunction that was selected.

[0022] For multi-sensor system 200, there may be variations in sensorreliability among the plurality of sensors 205 (e.g., based onvariations in SNR and other factors) during the tracking period suchthat adaptive processor 207 (when generating individual/system sensorreliability functions) may determine and assign a higher weight to abest performing sensor (with the highest SNR) than a (lower) weightassigned to a worse (or worst) performing sensor (e.g., with a lowerSNR) such that a fused result (combined reliability function for theplurality of sensors) may be weighted more towards the best performing(highest reliability) sensor. The variations in sensor reliabilities forthe plurality of sensors 205 may be caused by a number of factorsincluding weather conditions, different sensor attributes such as betterrange accuracy of an RF sensor than an IR sensor at longer ranges, orother factors causing at least one sensor to perform better than anothersensor during a predetermined tracking period.

[0023] Advantageously during operation as described herein, the SNR maybe used by adaptive processor 207 as a measure of sensor reliabilityduring a predetermined tracking period to help generate a sensorreliability function for each one of the plurality of sensors 205 usingeach one of the plurality of different fusion methods (shown in AppendixA). Thereafter, adaptive processor 207 may execute (perform) apredetermined algorithm incorporating additive, multiplicative, fuzzylogic (e.g., minimum-maximum), or other calculation (of each individualsensor reliability function) to generate at least one overall (combined)reliability function for the multi-sensor system (full plurality ofsensors). As part of generating the overall reliability function (forthe plurality of sensors) in accordance with the algorithm (process),adaptive processor 207 may adaptively weight (for a predetermined numberof frames) each sensor reliability function based on the SNR (a measureof individual sensor reliability or confidence level) for each sensorduring the tracking period.

[0024] However, under certain conditions (e.g., conditions causing afalse alarm rate above a predetermined threshold), the fused (combined)reliability result determined (generated) by adaptive processor 207 forthe (entire) plurality of sensors (during the tracking period) may notbe better than the individual sensor reliability result calculated fromthe performance of a better single sensor (e.g., the higher reliabilitysensor having the higher SNR). Therefore, adaptive processor 207 may useat least one additional predetermined sensor parameter (attribute) tobetter determine individual sensor reliability (function) weightingbased on whether or not a data fusion result (generated from each sensorcontributing) provides a more reliable result than a reliability resultfrom a (better performing) single sensor.

[0025] Relying on predetermined measurements and analysis (e.g., testingand/or computer simulation of sensor operation), adaptive processor 207may use the comparative (relative) received operating characteristics(ROC) between each sensor (for each different fusion method) as theadditional sensor parameter to help determine reliability weighting foreach one of the plurality of sensors 205 during a predetermined trackingperiod. The ROC performance (curve) for each one of the plurality ofsensors 205 may be generated (determined) using likelihood functions torepresent (characterize) sensor information (during target tracking)such as (target) detections, no detections, measured SNRs, and othersensor information obtained from sensor measurements, observations, orother sensor data outputs. Thereafter, the ROC likelihood function foreach sensor may be combined to generate likelihood (probability)functions of correct classification (recognition) of target and decoy(false target) for system 200.

[0026] Advantageously, the predetermined ROC performance (as shown inFIGS. 5, 6), predetermined calculations of other parameters (e.g.,critical false alarm rate for system 200), and other predeterminedfusion method results may be entered into a fusion table and input toadaptive processor 207 along with the other data (e.g., SNRmeasurements) from the plurality of sensors 205 to use duringperformance of the predetermined algorithms (to generate weightedreliability functions and data fusion results) executed by processors207, 208. For one example of the fusion table content, the fusedreliability performance (e.g., probability of correct classification oftarget and/or decoy) using additive fusion may be better than the singlesensor reliability performance when a critical false alarm rate is notreached (satisfied) as shown in FIG. 5c. This predetermined fusionmethod result may be part of the fusion table and used by adaptiveprocessor 207 for adaptive weighting (e.g., weighting the additivefusion result more) of the individual/combined sensor reliabilityfunctions. Another example of better additive fusion performance (put inthe fusion table input to adaptive processor 207) may be when thelikelihood function readings (values) are close to zero as occurs whenthe readings are from the tails of a bell-shaped likelihood function(for each sensor). For this exemplary embodiment, adaptive processor 207may assign (via additive fusion) a greater weight to the sensorcontributions from peak readings since readings from the peaks of thelikelihood functions are more reliable than the readings from the tails.For an accurate measure of the reliability weighting for thisembodiment, adaptive processor 207 may use the BPA (basic probabilityassignment) of the empty sets calculated from a predeterminedDempster-Shafer computation as the BPA of the empty sets is near onewhen the likelihood reading is near zero, and the BPA is near zero whenthe likelihood reading is near the peak of the likelihood function.

[0027] Also, using the predetermined Dempster-Shafer computation, SNRmay be calculated from a summation of ignorance sets as a measure of thenoise intensity. Additionally, in response to the fusion table input,fusion selection processor 208 may delay fusion and/or decision output210 if the measured values of the empty sets are low or the measuredvalues of the ignorance sets are high at a predetermined, particulartime frame (indicating a low confidence of correct classification forthe particular time frame). The predetermined Dempster-Shafercomputation is disclosed in the cross-referenced provisional applicationSer. No. 60/367,282, filed Mar. 26, 2002.

[0028] For multi-sensor system 200, generation of the likelihood(probability) functions for correct classification (P_(cc)) of targetand decoy using ROC likelihood function generation may includepredetermination of the likelihood function for individual sensor noisecaused by temporal fusion (diversity) as each sensor (auto) correlatesdata from multiple time frames (e.g., 120 time frames) during apredetermined tracking period. The temporal noise measurements (errors)for each one of the plurality of sensors 205 may be represented as arandom variable (RV) where the negative impact of RV may be reducedusing a plurality of methods including spatial and temporal fusionmethods (used to combine data from differently located sensors and/or asingle sensor outputting a plurality of data frames) to increase theprobability of correct classification for a target and/or decoy (P_(cc),P_(ct)). Spatial and temporal fusion methods may be used to generate acombined likelihood (pdf) function for differently located sensorsand/or sensors having a plurality of data time frames.

[0029] ROC (received operating characteristics) performance curves maybe generated using a plurality of methods including calculation of thecombined probability density function (pdf or likelihood function) forthe plurality of different fusion methods (likelihood functions shown inAppendix A). The methods may be based on a two-object (e.g., target—t,decoy—d), spatial fusion example (e.g., IR and RF sensor) where thelikelihood functions (representing P_(cc)) may be expressed as p(t1),p(d1) for a first sensor (sensor1—IR), and by p(t2), p(d2) for a secondsensor (sensor2—RF).

[0030] Alternatively, ROC curves may be generated using computersimulations (calculations) to generate a high number of random samples(e.g., 10,000) to represent RVs with different pdfs. Thereafter, thecombined pdfs may be determined from the histograms of combined RVsbased on the different fusion methods (shown in Appendix A). Exemplarydiagrams of ROC performance curves (generated using the alternativemethod) representing the probability of correct classification (versusprobability of false alarm) for the plurality of sensors 205 of system200 are shown in FIGS. 5a-5 h in accordance with an embodiment of thepresent invention. The probability of false alarm is the probabilitythat a target or decoy is detected when actually there is no target ordecoy (within the field of view of the plurality of sensors 205).

[0031]FIGS. 5a, 5 b show exemplary curves of the probability of correctclassification of decoy (P_(cd)—FIG. 5a) and target (P_(ct)—FIG. 5b)versus (vs.) probability of false alarm (P_(f)), respectively, forsystem 200 that may be generated using additive temporal fusion (shownin Appendix A) across two time frames of one of the sensors 205 (e.g.,an IR sensor). FIGS. 5e, 5 f show exemplary curves of the probability ofcorrect classification of decoy (P_(cd)—FIG. 5e) and target (P_(ct)—FIG.5f) versus (vs.) probability of false alarm (P_(f)), respectively, forsystem 200 that may be generated using multiplicative temporal fusion(shown in Appendix A) across two time frames of one of the sensors 205(e.g., an IR sensor). For FIGS. 5a, 5 b, 5 e, 5 f, the solid curves showthe P_(cc) performance of each individual time frame, and the dashedcurves show the fused P_(cc) performance across two time frames.

[0032] As shown in FIGS. 5a, 5 b, 5 e, 5 f, the fused Pcc performance(dashed curve) for both additive and multiplicative temporal fusion maybe better (probability of correct classification closer to 1) than theindividual time frame performance since the RVs in different time frameshave similar means and standard deviations.

[0033]FIGS. 5c, 5 d show exemplary curves of the probability of correctclassification of decoy (P_(cd)—FIG. 5c) and target (P_(ct)—FIG. 5d)versus (vs.) probability of false alarm (P_(f)), respectively, forsystem 200 that may be generated using additive spatial fusion (shown inAppendix A) between the plurality of sensors 205 (e.g., IR sensor and RFsensor). FIGS. 5g, 5 h show exemplary curves of P_(cd) (FIG. 5g) andP_(ct) (FIG. 5h) vs. P_(f), respectively, for system 200 that may begenerated using multiplicative spatial fusion (shown in Appendix A)between the plurality of sensors 205 (e.g., IR sensor and RF sensor).

[0034] For FIGS. 5c, 5 d, 5 g, 5 h, the solid curves show the P_(cc)(ROC) performance of a single IR sensor, the dot-dashed curves show theP_(cc) performance of a single RF sensor, and the dashed curves show thefused P_(cc) performance between the RF and IR sensors. The exemplarycurves shown in FIGS. 5c, 5 d, 5 g, 5 h may be generated using an RFsensor and an IR sensor, but it is noted that the selection of thesesensors is solely exemplary and should not be viewed as a limitationupon the invention.

[0035] As shown in FIGS. 5c, 5 d, 5 g, 5 h, the fused P_(cd) performance(dashed curve) for both additive and multiplicative fusion may be better(probability of correct classification closer to 1) than the singlesensor performance of either the RF or IR sensor where (as shown inFIGS. 5c, 5 g) the RF sensor is the better individual (single) sensor.Alternatively, as shown in FIGS. 5d, 5 h, the single RF sensorperformance (dot-dashed curve) may be better than either the fused Pctperformance for both additive and multiplicative fusion or the single IRsensor performance.

[0036] As shown in FIGS. 5d, 5 h, when the plurality of sensors 205 havevery dissimilar ROC performances (showing a large difference), the fusedROC (P_(cc)) performance may be worse than the better single sensor(e.g., RF) ROC performance, but still better than the worse singlesensor (e.g., IR) ROC performance which may indicate that the worsesingle sensor is negatively impacting (dragging down) the fused ROCperformance. In response to this situation, adaptive processor 207 mayassign less weight to the contribution (sensor reliability function)generated from the worse sensor (e.g., IR) such that the fused(combined) system reliability function generated by processor 207 isweighted more towards the contribution (sensor reliability function)generated from the better single sensor (e.g., RF) to improve systemreliability.

[0037] Also, as shown in FIGS. 5c, 5 g, the fused ROC performance isonly better than the better single sensor ROC performance when a Pfthreshold (calculated critical false alarm rate or probability of falsealarm threshold) is not reached (satisfied) as the fused and bettersingle sensor ROC performances curves reach substantially the same value(approximately a probability of 1) after this P_(f) threshold isreached. In response to this situation, adaptive processor 207 maygenerate a system (combined) reliability function based on each sensorSNR and an F/S ratio where the F/S ratio may represent the ratio betweenthe fused ROC performance and the better single sensor ROC performanceand be dependent on a false alarm rate (fa) function and the criticalfalse alarm rate.

[0038] Also, exemplary diagrams of fused ROC performance curves (alsogenerated using the alternate random sampling method) representing theprobability of correct classification (versus probability of falsealarm) for the plurality of sensors 205 of system 200 (using fourdifferent fusion methods) are shown in FIGS. 6a-6 d in accordance withembodiments of the present invention. The four different fusion methodsmay include additive, multiplicative, minimum (fuzzy logic), and maximum(fuzzy logic) fusion methods as shown in Appendix A.

[0039]FIGS. 6a, 6 b show exemplary curves of the probability of correctclassification of decoy (P_(cd)—FIG. 6a) and target (P_(ct)—FIG. 6b)versus (vs.) probability of false alarm (P_(f)), respectively, forsystem 200 that may be generated using all four different methods oftemporal fusion (shown in Appendix A) across two time frames of sensors205 (e.g., an IR sensor) for all four fusion methods. FIGS. 6c, 6 d showexemplary curves of P_(cd) (FIG. 6c) and P_(ct) (FIG. 6d) vs. P_(f),respectively, for system 200 that may be generated using all fourmethods of spatial fusion (shown in Appendix A) between the plurality ofsensors 205 (e.g., IR sensor and RF sensor).

[0040] For FIGS. 6a, 6 b, the solid curves show the fused P_(cc) (ROC)performance (of the single IR sensor) using additive temporal fusion,the dot-dashed curves show the fused P_(cc) performance using maximum(fuzzy logic) temporal fusion, the dashed curves show the fused P_(cc)performance using multiplicative temporal fusion, and the dotted curvesshow the fused Pcc performance using minimum (fuzzy logic) temporalfusion.

[0041] As shown in FIGS. 6a, 6 b, maximum temporal fusion is the worstamong the four temporal diffusion methods as its fused P_(cc)performance (dot-dashed curve) shows no improvement from the performanceof a single frame while the other three fusion methods (minimum,additive, and multiplicative) do show improvement across the two timeframes.

[0042] For FIGS. 6c, 6 d, the solid curves show the fused P_(cc) (ROC)performance between sensors 205 (e.g., an IR and RF sensor) usingadditive spatial fusion, the dot-dashed curves show the fused P_(cc)performance using maximum (fuzzy logic) spatial fusion, the dashedcurves show the fused P_(cc) performance using multiplicative spatialfusion, and the dotted curves show the fused P_(cc) performance usingminimum (fuzzy logic) spatial fusion.

[0043] As shown in FIG. 6c, the fused P_(cd) performances formultiplicative and maximum fusion methods are better (probability closerto 1) than additive fusion. Alternatively, as shown in FIG. 6d, thefused P_(ct) performance for additive fusion is better than themultiplicative and maximum fusion methods.

[0044] In accordance with embodiments of the present invention,(simulated) results of the four different temporal fusion algorithms (asshown in Appendix A) performed by multi-sensor system 200 are shown inFIG. 3. FIGS. 3a, 3 b show diagrams of exemplary IR sensor reliabilityfunctions of P_(ct) and P_(cd) for temporal fusion across four timeframes of target and decoy performance data, respectively. As shown inFIGS. 3a, 3 b, the maximum temporal fusion performs worse than the otherthree fusion methods. FIGS. 3c, 3 d show diagrams of exemplary RF sensorreliability functions of P_(ct) and P_(cd) for temporal fusion acrossfour time frames of target and decoy performance data, respectively. Asshown in FIGS. 3c, 3 d, the RF sensor is the better sensor (as comparedto the IR sensor in FIGS. 3a, 3 b) as only two time frames of temporalfusion (integration) may be required to increase the P_(ct) to 100%, andonly four time frames of temporal integration may be required toincrease the P_(cd) to 100%.

[0045] Also, in accordance with embodiments of the present invention,(simulated) results of additive and multiplicative spatial fusionalgorithms (as shown in Appendix A) performed by multi-sensor system 200for plurality of sensors 205 (e.g., IR and RF sensor) are shown in FIG.4. FIG. 4 includes the P_(ct), P_(cd), P_(f), and P_(mc) (probability ofmisclassification) probabilities during low and high SNR conditions forboth fusion methods. As shown in FIGS. 4a, 4 b for low SNR conditionsusing additive fusion, the P_(ct) may be improved to 95% (as compared toP_(ct) of 80% with prior art classification schemes that fail to usetemporal/spatial fusion), and the P_(cd) may be improved to 88% (ascompared to P_(cd) of 76% with prior art classification schemes). Also,as shown in FIGS. 4b, 4 d for low SNR conditions, the fused performance(Pct of 95% and 96% for additive and multiplicative fusion,respectively) is worse than the performance of the better single RFsensor (P_(ct) of 98% as predetermined using the random samplingmethod). Also, as shown in FIGS. 4a, 4 c, even for high SNR conditionsthe fused P_(ct) performance (P_(ct) of 97% for both additive andmultiplicative fusion) is worse than the performance of the bettersingle RF sensor (P_(ct) of 98% as predetermined using the randomsampling method). Alternatively, as shown in FIGS. 4a, 4 c with highSNR, the fused P_(cd) performance (P_(cd) of 96% and 97% for additiveand multiplicative fusion, respectively) is better than the performanceof the better single RF sensor (P_(cd) of 92% as predetermined using therandom sampling method).

[0046] As described herein, the sensor classification results shown inFIGS. 3-6 may be part of the fusion table input to processors 207, 208to help adaptively determine/select the best fusion method, adaptivelyweight different sensor contributions based on individual sensorreliability functions, and adaptively select and execute a fusion methodor predetermined combination of multiple fusion methods during thecurrent time frame or delay data fusion to a future time frame based onpredetermined criteria including flight conditions and situations,reliabilities of different sensors, weather conditions, SNR, range,classification confidence, or other criteria.

[0047] A plurality of advantages may be provided in accordance withembodiments of the present invention including a data fusion selectionmethod that adaptively weights the contributions from different sensors(within a multi-sensor system) using multiple fusion methods to generatea plurality of different system reliability functions (wherein one or apredetermined combination is selected in accordance with predeterminedcriteria) where SNR and relative ROC performance between sensors may beused as measures of reliability.

[0048] Although the invention is primarily described herein usingparticular embodiments, it will be appreciated by those skilled in theart that modifications and changes may be made without departing fromthe spirit and scope of the present invention. As such, the methoddisclosed herein is not limited to what has been particularly shown anddescribed herein, but rather the scope of the present invention isdefined only by the appended claims.

What is claimed is:
 1. A method for integrating data received from aplurality of sensors, comprising: receiving data from a plurality ofsensors; determining a S/N (signal-to-noise) ratio for each sensor basedon signal measurements of the received data; determining a plurality ofreliability functions for the plurality of sensors from using aplurality of predetermined calculations combining each sensorreliability function which are determined and individually weightedbased on said S/N ratio for each sensor and differences betweenpredetermined operating characteristics for each sensor; and selectingone or a predetermined combination of said plurality of reliabilityfunctions as a current reliability function based on the selected one orpredetermined combination satisfying predetermined thresholds.
 2. Themethod of claim 1, further comprising: determining to delay saidselecting for a predetermined time period based on said selected atleast one failing to satisfy said predetermined threshold.
 3. The methodof claim 1, wherein said determining a plurality of reliabilityfunctions includes determining said plurality of reliability functionsfrom using one of a predetermined additive, multiplicative, and fuzzylogic calculation combing each sensor reliability function.
 4. Themethod of claim 3, wherein said determining a plurality of reliabilityfunctions includes determining empty sets and ignorance sets for thereceived data from the plurality of sensors based on said predeterminedmultiplicative calculation.
 5. The method of claim 4, wherein saidpredetermined multiplicative calculation includes at least a modifiedportion of a Dempster's data fusion algorithm.
 6. The method of claim 1,wherein said determining a plurality of reliability functions includesdetermining said individual weighting based on a predetermined parameterfor the plurality of sensors satisfying a predetermined threshold, anddetermining said plurality of reliability functions, based on using saidplurality of predetermined calculations for a single sensor reliabilityfunction, when said predetermined parameter fails to satisfy saidpredetermined threshold.
 7. The method of claim 6, wherein saidpredetermined parameter is a false alarm rate for the plurality ofsensors.
 8. The method of claim 1, wherein said determining a pluralityof reliability functions includes determining said plurality ofreliability functions based on differences between probabilities ofclassification for each sensor as compared to a single sensor having thehighest probability of classification.
 9. The method of claim 1, whereinsaid plurality of sensors includes at least one of a laser, IR(infrared), and RF (radio frequency) sensor.
 10. The method of claim 1,wherein said determining a plurality of reliability functions includesdetermining said plurality of reliability functions based on determininga single sensor, as compared to the plurality of sensors, as having thebest performance for at least one predetermined sensor parameter. 11.The method of claim 10, wherein said at least one predetermined sensorparameter is one of operating characteristics and S/N ratio for saidsingle sensor.
 12. The method of claim 11, wherein said single sensor isone of a laser, IR, and RF sensor.
 13. The method of claim 1, whereinsaid selecting includes selecting said current reliability function toincrease the probability of classifying one of a target and decoy abovea predetermined threshold.
 14. A multi-sensor system, comprising: aplurality of sensors for receiving data; and at least one controller forperforming the steps of: determining a S/N (signal-to-noise) ratio foreach sensor based on signal measurements of the received data;determining a plurality of reliability functions for the plurality ofsensors from using a plurality of predetermined calculations combiningeach sensor reliability function which are determined and individuallyweighted based on said S/N ratio for each sensor and differences betweenpredetermined operating characteristics for each sensor; and selectingone or a predetermined combination of said plurality of reliabilityfunctions as a current reliability function based on the selected atleast one satisfying a predetermined threshold.
 15. The system of claim14, wherein said controller to determine said individual weighting basedon a predetermined parameter for the plurality of sensors satisfying apredetermined threshold, and said controller to determine said pluralityof reliability functions, based on using said plurality of predeterminedcalculations for a single sensor reliability function, when saidpredetermined parameter fails to satisfy said predetermined threshold.16. The system of claim 15, wherein said predetermined parameter is afalse alarm rate for the plurality of sensors.
 17. The system of claim14, wherein said plurality of sensors includes at least one of a laser,IR (infrared) sensor, and RF (radio frequency) sensor.
 18. Amachine-readable medium having stored thereon a plurality of executableinstructions, the plurality of instructions comprising instructions to:receive data from a plurality of sensors; determine a S/N(signal-to-noise) ratio for each sensor based on signal measurements ofthe received data; determine a plurality of reliability functions forthe plurality of sensors from using a plurality of predeterminedcalculations combining each sensor reliability function which aredetermined and individually weighted based on said S/N ratio for eachsensor and differences between predetermined operating characteristicsfor each sensor; and select at one or a predetermined combination ofsaid plurality of reliability functions to use as a current reliabilityfunction based on the selected at least one satisfying a predeterminedthreshold.
 19. The medium of claim 18, wherein said instructions todetermine a plurality of reliability functions include instructions todetermine said individual weighting based on a predetermined parameterfor the plurality of sensors satisfying a predetermined threshold. 20.The medium of claim 18, wherein said instructions to determine aplurality of reliability functions include instructions to determinesaid plurality of reliability functions based on differences betweenprobabilities of classification for each sensor as compared to a singlesensor having the highest probability of classification.
 21. The mediumof claim 18, wherein said instructions to determine a plurality ofreliability functions include instructions to determine said pluralityof reliability functions from using one of a predetermined additive,multiplicative, and fuzzy logic calculation combining each sensorreliability function.
 22. The medium of claim 18, wherein saidinstructions to select include instructions to select said currentreliability function to increase the probability of classifying one of atarget and a decoy above a predetermined threshold.